**** BEGIN LOGGING AT Tue Dec 16 17:46:44 2008

Dec 16 17:46:45 *	Now talking on #ooonlc
Dec 16 17:46:45 *	Topic for #ooonlc is: OpenOffice.org Native Language Confederation users channel | http://wiki.services.openoffice.org/wiki/IRC_Meetings |Tuesday, December 16th at 5:00 pm UTC: Meeting on QA of l10n builds | *TODAY*
Dec 16 17:46:45 *	Topic for #ooonlc set by MechtiIde at Tue Dec 16 09:59:07 2008
Dec 16 17:47:12 khirano	Hi, JiHui :)
Dec 16 17:49:41 khirano	JiHui: What time is it now overthere in Korea?
Dec 16 17:49:45 JiHui	I'm little tired. 'cause I waked at 6:00 am. and now 1:50 am -_-;
Dec 16 17:49:55 JiHui	it's same. +9 GMT
Dec 16 17:50:01 khirano	JiHui: I see 
Dec 16 17:50:30 JiHui	In fact, we use the japanese time. maybe tokyo time?
Dec 16 17:51:01 JiHui	we're in tokyo time zone.
Dec 16 17:52:01 khirano	JiHui: Yeh, the same, 1:51, it's Seoul time :)
Dec 16 17:52:41 JiHui	right. :)
Dec 16 17:53:19 *	thorstenziehm (n=chatzill@nat/sun/x-a0e0ed945e7c1228) has joined #ooonlc
Dec 16 17:53:35 thorstenziehm	hi all
Dec 16 17:54:01 khirano	Hi TZ
Dec 16 17:54:33 JiHui	Hi thorstenziehm
Dec 16 17:54:58 *	RQ (n=rq@78.59.136.38) has joined #ooonlc
Dec 16 17:55:05 RQ	hi
Dec 16 17:55:32 khirano	Hi RQ
Dec 16 17:56:04 RQ	so, do I get to tell all my complaints here now? :D
Dec 16 17:56:10 RQ	(about qa)
Dec 16 17:57:44 khirano	RQ: could you introduce yourself?  My name is Kazunari Hirano from Japanese Language Project? Nice to see you here :)
Dec 16 17:57:44 JiHui	RQ, maybe you can. But I'm not sure you'll be able to get any answer.
Dec 16 17:58:16 RQ	khirano, i'm Rimas Kudelis, the Lithuanian QA self-assigned lead :)
Dec 16 17:58:53 RQ	nice to meet you :)
Dec 16 18:00:16 rafaella	Hi Everybody!
Dec 16 18:00:31 khirano	Hi Rafaella
Dec 16 18:00:40 MechtiIde	hello
Dec 16 18:00:56 ain	hi
Dec 16 18:01:03 RQ	yo!
Dec 16 18:01:33 JiHui	hello rafaella, MechtiIde, ain :)
Dec 16 18:01:49 rafaella	it think that we can start the meeting, right?
Dec 16 18:01:49 *	MechtiIde is now known as Mechtilde
Dec 16 18:01:50 *	Stefan___b_away is now known as Stefan___b
Dec 16 18:01:53 Stefan___b	Hi, all!
Dec 16 18:02:19 JiHui	Hi, Stefan
Dec 16 18:02:39 khirano	Rafaella: let us start.
Dec 16 18:02:51 *	thackert (n=thackert@dslb-092-072-230-203.pools.arcor-ip.net) has joined #ooonlc
Dec 16 18:03:02 rafaella	ok. There are 2 topics to be discussed today
Dec 16 18:03:02 thackert	Hello @ll :)
Dec 16 18:03:22 RQ	heh, perhaps we should wait another 7 minutes
Dec 16 18:03:31 rafaella	how to do release QA - and how to release a localized build
Dec 16 18:04:16 rafaella	2) how to improve l10n testing between releases?
Dec 16 18:05:05 *	paveljanik (n=Pavel@unaffiliated/paveljanik) has joined #ooonlc
Dec 16 18:05:13 paveljanik	Moin
Dec 16 18:05:15 Stefan___b	Hi, Pavel!
Dec 16 18:05:25 thackert	Hi paveljanik :)
Dec 16 18:05:53 Stefan___b	paveljanik: rafaella just named the 2 topics (see alos http://wiki.services.openoffice.org/wiki/IRC_Meetings#Logistics)
Dec 16 18:06:13 rafaella	There have been a couple of quesitons also that have been raised by Sankarshan and 1 quesiton from Ain to discuss some issues in case we have time
Dec 16 18:06:30 *	sophi silently says Hi to all :)
Dec 16 18:06:44 RQ	hi sophi 
Dec 16 18:06:57 thackert	Hi sophi :)
Dec 16 18:07:07 *	artietee (n=arthur@cp431174-a.dbsch1.nb.home.nl) has joined #ooonlc
Dec 16 18:07:24 rafaella	for the ones who just joined - hi everybody - let's start talking about the first topic
Dec 16 18:07:31 artietee	good evening all
Dec 16 18:08:00 rafaella	wow - good to see you all here!
Dec 16 18:08:54 rafaella	Regarding topic 1) how to do release QA - and how to release a localized build - questions?
Dec 16 18:09:18 thackert	Not at the moment ...
Dec 16 18:09:26 RQ	release QA is those sanity tests, right?
Dec 16 18:09:46 Stefan___b	rafaella: any URL, isssues or mails serving our subject that might be worth should read "beside"?
Dec 16 18:09:51 Stefan___b	RQ: Yes.
Dec 16 18:09:59 RQ	hm
Dec 16 18:10:06 RQ	then i have one problem with those
Dec 16 18:10:15 Stefan___b	tell :-)
Dec 16 18:10:30 RQ	the testers are told to download the TestTool from CVS
Dec 16 18:10:55 RQ	why can't we just download some zip archive from somewhere?
Dec 16 18:11:22 Mechtilde	you can do it RQ
Dec 16 18:11:30 RQ	link?
Dec 16 18:11:48 Mechtilde	http://ooopackages.good-day.net/pub/OpenOffice.org/qa/
Dec 16 18:11:53 rafaella	just one moment: let's define if we are talking about Sanity testing which is on RC builds or if we are talking about regular testing...
Dec 16 18:12:16 RQ	I'm talking about RC
Dec 16 18:12:48 RQ	Mechtilde, so, they just have to download one of the testautomotion packages, right?
Dec 16 18:13:01 thorstenziehm	RQ : do you start testing on RC only?
Dec 16 18:13:10 rafaella	RQ, do you use TestTool to test l10n RC builds?
Dec 16 18:13:37 RQ	rafaella, in sanity test scenario there are two tests that explicitly involve testtool
Dec 16 18:14:15 RQ	so i guess I have to use the testtool to test l10n RC builds, yes 
Dec 16 18:14:45 RQ	or am I wrong?
Dec 16 18:15:13 Stefan___b	RQ: Yes. TestTool is the best to cover a lot of Office code in a short time.  
Dec 16 18:15:18 sophi	RQ: you're right, they are at the end of sanity checks
Dec 16 18:15:38 RQ	thorstenziehm, actually I'm new to testing â I joined it after I learned why Lithuanian 3.0 builds are again in extended/ and not in localized/
Dec 16 18:15:48 rafaella	RQ, no, you are not wrong. I was wondering which tests you were referring to. Yes, the sanity check scenario is recommended: http://wiki.services.openoffice.org/wiki/Sanity_Check_Of_L10n_Builds
Dec 16 18:16:01 Stefan___b	In localized builds, build errors are the most likely thing to happen. 
Dec 16 18:16:03 RQ	rafaella, wait a sec, I'll give you the numbers
Dec 16 18:16:24 Stefan___b	And build errors are very quickly detected by testtool.
Dec 16 18:17:08 *	Thalion72 (n=sca@p3EE29D4F.dip0.t-ipconnect.de) has joined #ooonlc
Dec 16 18:17:14 sophi	Stefan___b: yes but results are quiete long to analyse and need some knowledge
Dec 16 18:17:16 thorstenziehm	Stefan___b : but general testing should be finished before RC and then sanity tests should be used for the release
Dec 16 18:17:57 thorstenziehm	sophi : this shouldn't be the case for the next releases
Dec 16 18:18:07 Stefan___b	thorstenziehm: Yes.
Dec 16 18:18:14 thorstenziehm	the results of the test should be zero errors and zero warnings
Dec 16 18:18:24 RQ	rafaella, 110029, 110031 and 110032 at least
Dec 16 18:18:29 Thalion72	Hi 
Dec 16 18:18:37 thackert	Hi Thalion72 :)
Dec 16 18:18:39 Mechtilde	thorstenziehm, but this is the problem
Dec 16 18:18:40 RQ	rafaella, they are a part of the release sanity scenario
Dec 16 18:18:40 thorstenziehm	if this isn't the case for your language and your platform, than you have to write an issue to the automation team
Dec 16 18:18:43 Thalion72	thorstenziehm: nice theory .. but we should not rely on this
Dec 16 18:18:47 RQ	and they require the testtool
Dec 16 18:19:12 thorstenziehm	this isn't theory! this will be reality in the near future, when you are working with us
Dec 16 18:19:35 thorstenziehm	we need the feedback, qualified feedback and not "it doesn't work" 
Dec 16 18:20:06 Thalion72	thorstenziehm: I'm working with you - have been for quite a while. And things improve .. but we are still far away from "zero errors" in a community environment
Dec 16 18:20:29 rafaella	RQ: thanks. From your experience how long does it take to run 110029, 110031 and 110032? Because my unserdanding was that it should not take more than 4 hours to run a Sanity check scenario.
Dec 16 18:20:39 thorstenziehm	then we need specific environments -> perhaps TestBots 
Dec 16 18:20:41 *	JiHui has quit (Read error: 113 (No route to host))
Dec 16 18:20:42 Thalion72	so - yes, we should give this fedback, I agree .. but we cannot give this wehn we are busy doing release testing
Dec 16 18:21:35 RQ	rafaella, dunno, i still haven't installed tortoiseCVS on the windows box I performed tests on
Dec 16 18:21:38 thorstenziehm	Thalion72 : therefore the release testing should be minimized in my opinion
Dec 16 18:21:44 Mechtilde	rafaella, if you have experience it might be
Dec 16 18:21:56 *	JiHui (n=X-iRc@211.228.29.125) has joined #ooonlc
Dec 16 18:22:00 Mechtilde	otherwise you need more time than 4 hours
Dec 16 18:22:01 Thalion72	thorstenziehm: yes
Dec 16 18:22:21 thorstenziehm	inside Sun we are using for RC testing only automated processes, I think this could be the L10N teams also.
Dec 16 18:22:45 Thalion72	thorstenziehm: I yould not suggst this - not for new teams
Dec 16 18:22:56 rafaella	Thalion72, Mechtilde we need to know if this is not the right solution....
Dec 16 18:23:15 sophi	thorstenziehm: no, the choice should be left to the team
Dec 16 18:23:23 Thalion72	thorstenziehm: new teams should focus on some very basic manual tests 
Dec 16 18:23:47 thorstenziehm	sophie : I agree with you. But this work only in good structured L10N teams.
Dec 16 18:23:49 thackert	sophi: +1
Dec 16 18:23:51 rafaella	thorstenziehm, my team is also performing manual sanity check on RC builds .. so it's not all automated
Dec 16 18:24:00 Thalion72	thorstenziehm: if the teams "advance" they should have the chance to go with automated tests
Dec 16 18:24:01 Mechtilde	rafaella, the problem is there is a different need of time for the experience teams and the new teams
Dec 16 18:24:10 sophi	and some members will not want to do automated testing, others will want
Dec 16 18:24:53 rafaella	Mechtilde, sophi this is why we need to provide different solutions based on experience, time and resources....
Dec 16 18:25:09 RQ	on my team, only one tester has ran those testtool tests so far
Dec 16 18:25:19 thorstenziehm	In how many languages OOo 3.0 is translated and how many are released until now. I do not know, but I think the number differ extremely. So how could we held to release more languages?
Dec 16 18:25:19 sophi	rafaella: yes I agree, we should let the chance for all participations
Dec 16 18:25:23 RQ	the linux guy
Dec 16 18:25:45 Stefan___b	sophi: There is always room and need for both manual and automated testing.
Dec 16 18:26:09 sophi	Stefan___b: yes, so the choice should be left :)
Dec 16 18:26:25 Stefan___b	And seperation should be: Leave the boring repeating stuff to machines :-)
Dec 16 18:26:44 rafaella	sophi, some time ago Petr Dedacek wrote a short guideline http://wiki.services.openoffice.org/wiki/Sanity_Check_Of_L10n_Builds. There it says which are the really basic requirements to do a short sanity check
Dec 16 18:26:55 sophi	thorstenziehm: more communication, more questions and ask session, more presence will help 
Dec 16 18:27:02 RQ	hm
Dec 16 18:27:06 RQ	by the way
Dec 16 18:27:14 Thalion72	Stefan___b: yes .. but you need someone to start the automation and someone to check the results .. these are all barriers for new testers
Dec 16 18:27:28 RQ	is there a reason why automated tests have to be performed by l10n teams?
Dec 16 18:27:56 sophi	RQ: not at all, for FR team, they are different users
Dec 16 18:28:04 Stefan___b	sophi: You and me cover 80% in QA IRC a day because we say "Hello" once a day. There is room and I am there all day while working :-)
Dec 16 18:28:15 Thalion72	RQ: afaik . no. There is only one reason, why they are in the release sanity scenario : because I put them there
Dec 16 18:28:27 *	KAMI_Work (n=szalaik@helpdesk.navigatorrt.hu) has joined #ooonlc
Dec 16 18:28:37 KAMI_Work	Hey Folks!
Dec 16 18:28:42 Thalion72	Hi KAMI_Work
Dec 16 18:28:56 KAMI_Work	Did I miss something?
Dec 16 18:28:57 thackert	Hi KAMI_Work
Dec 16 18:29:07 sophi	Stefan___b: we need to tell that to all other teams :)
Dec 16 18:29:23 Stefan___b	sophi: Please go ahead :-)
Dec 16 18:30:02 sophi	Stefan___b: I'll do, but this session is also here for that, better communication between us 
Dec 16 18:30:20 rafaella	Thalion72, wouldn't it be better to have 2 sanity scheck scenario: 1 basic and one with the automated tests?
Dec 16 18:30:53 rafaella	The teams would pick up the scenario that better suits them....
Dec 16 18:31:18 Thalion72	rafaella: we can setup 2 scenarios yes
Dec 16 18:31:49 sophi	rafaella: the team who don't know will pick up every thing, so yes, separate them is a good idea
Dec 16 18:31:50 RQ	Thalion72, sophi, then why can't we let some experienced team or even machines perform all automated tests for all locales?
Dec 16 18:32:01 Thalion72	but maybe some info about other teams work are more helpfull
Dec 16 18:32:22 JiHui	in my case, I hear about automated test at here. So I'd like to know what is automate test, or how to use the tool.
Dec 16 18:32:45 sophi	RQ: and who will check the results in your language ?
Dec 16 18:32:50 JiHui	and I want you to tell me about that. However, whether I use the tool or not is depended on me.
Dec 16 18:32:52 RQ	what's the point in downloading 100+MB of data just to run a bat file, if this could be done automatically?
Dec 16 18:32:57 Stefan___b	Since the TesTool is not a one-click-wonder (YET, we are constantl working on it) there must be a way to release a build manually.
Dec 16 18:33:03 RQ	sophi, what ARE the results?
Dec 16 18:33:13 JiHui	I think it's the same for all teams.
Dec 16 18:33:21 Thalion72	JiHui: there is a basic descritpion in the wiki: http://wiki.services.openoffice.org/wiki/VCLTesttool
Dec 16 18:33:23 rafaella	To me it seems that we need to set up a TestTool training session on how to use the testTool and on how best to evaluate the testing results
Dec 16 18:33:24 sophi	RQ: the .res files that you need to check for errors
Dec 16 18:33:36 RQ	sophi, are these text files, or images?
Dec 16 18:33:43 Mechtilde	RQ, checking the results doesn't work automatically
Dec 16 18:33:49 JiHui	thanks, Thalion72. I saw the page just few minutes ago. :)
Dec 16 18:33:58 sophi	RQ: text files that you read through the testtool
Dec 16 18:34:12 RQ	hm...
Dec 16 18:34:18 KAMI_Work	rafaella: It is a great idea
Dec 16 18:34:41 KAMI_Work	at least we need a detailed documentation of it
Dec 16 18:34:44 Mechtilde	rafaella, and a session to setup the testtool
Dec 16 18:34:50 KAMI_Work	if it is available
Dec 16 18:34:50 Stefan___b	The result files should be verified automatically. Reading them manually is only half the way to use a TOOL.
Dec 16 18:34:58 RQ	well, it's hard to comment while I haven't seen the testtool or its results. But I suspect that I would much rather just go through the results then take care to produce them first
Dec 16 18:35:08 Stefan___b	And with QUASTE, we are on the right way.
Dec 16 18:35:10 Mechtilde	Stefan___b, this is the theory
Dec 16 18:35:17 Thalion72	KAMI_Work: hmm .. there is very detailed documentation available .. but there is still need for a workshop
Dec 16 18:35:18 rafaella	thorstenziehm, would it be possible to organize a training session?
Dec 16 18:35:31 JiHui	KAMI_Work, right. we allow evryone to know there is another option, automated test.
Dec 16 18:35:48 RQ	rafaella, I suggest we should make downloading testtool easier first
Dec 16 18:35:48 Thalion72	Stefan___b: QUASTE is the wrong way .. at the moment
Dec 16 18:36:11 thorstenziehm	I don't know, what should be trained. Perhaps you can open a WIki page, where the questions can be collected and then it will be possible to organize a session, I think.
Dec 16 18:36:14 RQ	then the session is also useful, but I think it would be even more useful if it were recorded as a short youtube movie or something like that
Dec 16 18:36:20 Stefan___b	You know how Helge is doiing his best :-) Internally, the QUASTE thing runs and saves a lot of time already.
Dec 16 18:36:40 rafaella	RQ, a tutorial?
Dec 16 18:36:41 thorstenziehm	Thalion72 : without QUASTE it isn't possible to evaluate the results.
Dec 16 18:36:51 Thalion72	thorstenziehm: as I did some of such training sessions - people just need to be "guided" throug the first steps normally
Dec 16 18:36:52 Stefan___b	And still, in our network we already have the problem of tests running "inconsistently".
Dec 16 18:37:09 Thalion72	thorstenziehm: if no one else did already upload a set of test results
Dec 16 18:37:13 RQ	rafaella, well, something like "this is the test tool, you do this, and you get this, and in results you seek for this"
Dec 16 18:37:16 thorstenziehm	What are the first steps. Reading the manual?
Dec 16 18:37:18 Mechtilde	Stefan___b, evaluate the tests with zero errors
Dec 16 18:37:22 Mechtilde	and the rest?
Dec 16 18:37:40 Stefan___b	Yes. Red and orange are the things to deal with!
Dec 16 18:37:53 rafaella	thorstenziehm, do you know why it seems that downloading the testtoll is not that easy?
Dec 16 18:38:14 RQ	rafaella, I think I know.
Dec 16 18:38:52 RQ	rafaella, the only guy who went through CVS checkouting and installing it on Linux, wrote a small tutorial to our testers list
Dec 16 18:38:54 Stefan___b	Sorry folks... The Testtool itself may be introduced and discussed on another occasion...?
Dec 16 18:39:22 RQ	it was like "you have to install CVS, then you have to download the testtool and put it into CERTAIN directory, only then it will work"
Dec 16 18:39:28 RQ	where's the .exe installer?..
Dec 16 18:39:31 thorstenziehm	Rafaella : The TestTool is an environment and not a Tool. it base on the application 'VCLTestool' and scripts which changed from one milestone to the next (when feature and funcitonality changes come in)
Dec 16 18:40:16 thorstenziehm	RQ : It isn't possible for an environment. 
Dec 16 18:40:22 RQ	thorstenziehm, but why can't I download that environment just like I downloaded OOo?
Dec 16 18:40:29 Stefan___b	As I said above: Not a one-click-wonder YET :-)
Dec 16 18:40:46 thorstenziehm	OOo is an Application and not an environment!
Dec 16 18:41:13 Thalion72	thorstenziehm / Stefan___b: I'd really sugest to discuss testtool use in another session. The current discussion only proves one thing: using testtool is complicated for many of our community members
Dec 16 18:41:20 RQ	thorstenziehm, come on... if I can cvs checkout it, then I could just aswell download it
Dec 16 18:41:40 JiHui	wait.. if VCLT is provided by cvs, for use it, we just need a cvs client, don't we?
Dec 16 18:41:41 thorstenziehm	RQ : currently not!
Dec 16 18:41:45 rafaella	thorstenziehm, then in order to make it easier for the testers: is it possible to set up a testing environment and provide the testers with the rights to access that environment?
Dec 16 18:41:58 sophi	RQ: the tool in itself is in OOo, but not the test files
Dec 16 18:42:34 RQ	sophi, yeah, that's another mistery.. I chose to install the "testtool" component, but I didn't find the bat files mentioned in the tests
Dec 16 18:42:44 Mechtilde	http://ooopackages.good-day.net/pub/OpenOffice.org/qa/ here we can get the test files
Dec 16 18:43:08 RQ	Mechtilde, well, which of those files do I download?
Dec 16 18:43:15 Mechtilde	but there are not the scripts to run it automatically
Dec 16 18:43:15 *	karunakar (n=karunaka@61.11.19.23) has joined #ooonlc
Dec 16 18:43:32 RQ	doh
Dec 16 18:43:33 Mechtilde	RQ, on win the zip
Dec 16 18:43:48 RQ	Mechtilde, does it have the bat files?
Dec 16 18:44:06 Mechtilde	RQ, not at this time
Dec 16 18:44:15 JiHui	no. I can't find any excutable file in it
Dec 16 18:44:24 Mechtilde	thorstenziehm, and this is a big big problem
Dec 16 18:44:26 RQ	Mechtilde, but I need bat files to perform tests 10031 and 10032 ;)
Dec 16 18:44:45 thackert	JiHui: You do not need any executable ... ;)
Dec 16 18:44:52 Mechtilde	RQ, for these tests you only need *.bas fils
Dec 16 18:45:02 thackert	You need the testtool from OOo to run them ...
Dec 16 18:45:05 Thalion72	RQ: bat files will be there from m39 on .. but you still need a proper setup
Dec 16 18:45:15 Mechtilde	RQ there are in the download
Dec 16 18:45:15 RQ	oh.. my faults
Dec 16 18:45:18 RQ	it's bas, not bat
Dec 16 18:45:19 RQ	:)
Dec 16 18:45:52 RQ	ok, then back to 110029. that is scary
Dec 16 18:45:59 RQ	it says "read the documentation"
Dec 16 18:46:05 RQ	I don't have time or will for that... :)
Dec 16 18:46:09 thorstenziehm	I think we should stop here the bashing on the TestTool (environment). The questions/problems should be collected on a Wiki page and we will work such issues. There isn't a one click solution possible until now and not in the near future.
Dec 16 18:46:18 thorstenziehm	But we will try our best!
Dec 16 18:46:55 KAMI_Work	yes It should
Dec 16 18:47:16 Thalion72	thorstenziehm: agreed .. but until this thime we should not enforce people to use testtool
Dec 16 18:47:17 KAMI_Work	Also you can provide easy to install packages for this environment
Dec 16 18:47:30 artietee	thorstenziehm: right, thank you - let's move on 
Dec 16 18:47:31 RQ	hm
Dec 16 18:47:52 RQ	I think we could just have the line removed from 110029 which suggests CVS 
Dec 16 18:48:02 thorstenziehm	Thalion72 : the sanity check Rafaella provided doesn't say anything from TestTool.
Dec 16 18:48:05 RQ	the testtool page offers binary downloads
Dec 16 18:48:19 RQ	so it seems like I don't really need cvs to download the testtool
Dec 16 18:48:21 Mechtilde	RQ, 110029 These are Infos how to do testtool
Dec 16 18:48:27 sophi	Thalion72: could you update the TCM and make them optional for now?
Dec 16 18:48:30 Thalion72	thorstenziehm: but the TCM release sanity scenario still does ;)
Dec 16 18:48:38 Thalion72	sophi: I'll do
Dec 16 18:48:48 RQ	Mechtilde, yeah, and it says "make shure you have the most recent testscripts available (checkout for OOo cvs using qa/qatesttool)"
Dec 16 18:48:51 sophi	Thalion72: thanks :)
Dec 16 18:48:52 thorstenziehm	So use this http://wiki.services.openoffice.org/wiki/Sanity_Check_Of_L10n_Builds as sanity check and try to use TestTool for general testing between the release cycles
Dec 16 18:49:03 RQ	Mechtilde, can't I download the files from http://qa.openoffice.org/ooQAReloaded/AutomationTeamsite/ooQA-TeamAutomationBin.html instead?
Dec 16 18:49:18 thorstenziehm	I am not the maintainer for TCM, this is part of the L10N teams. So they have to decide which sanity check should be used.
Dec 16 18:49:32 Mechtilde	RQ, the first thing Thalion72 will change
Dec 16 18:49:43 Mechtilde	RQ, yes you can
Dec 16 18:49:43 RQ	Mechtilde, which one?
Dec 16 18:49:48 rafaella	Thalion72, the instructions say explicitely that depending on time resource experience you should go one way or the other...
Dec 16 18:49:56 RQ	ok, so that was my showstopper
Dec 16 18:50:05 Stefan___b	RQ: It must be possible to release without TestTool. AFTER release, it is recommended to get familiar with theTestTool and its "specialities" that definately take time to get dealt with.
Dec 16 18:50:22 Thalion72	Stefan___b: +1
Dec 16 18:50:25 rafaella	Can we see if there is anything else which need to be discussed related to the topic how to do release QA - and how to release a localized build
Dec 16 18:51:02 Stefan___b	Can we agree on the updated pages as a result?
Dec 16 18:51:04 Thalion72	did we already speak about the issues you need to file for distribution?
Dec 16 18:51:07 Mechtilde	rafaella, what is the technical process to release
Dec 16 18:51:36 rafaella	1) run sanity check on RC builds
Dec 16 18:51:50 Mechtilde	how to document it
Dec 16 18:51:55 KAMI_Work	thorstenziehm: for sanity test
Dec 16 18:52:10 Mechtilde	http://qatrack.services.openoffice.org/
Dec 16 18:52:39 RQ	hm
Dec 16 18:52:40 RQ	ok
Dec 16 18:52:41 rafaella	2) update the test status on QATrack
Dec 16 18:52:53 RQ	where do I provide feedback on TCM?
Dec 16 18:53:02 KAMI_Work	I saw a comjplete website @ mozilla where the tester is able to check the test step by step and select if one functionality is working or not
Dec 16 18:53:34 RQ	yes
Dec 16 18:53:38 RQ	it's called Litmus
Dec 16 18:53:52 rafaella	RQ, to the dev@l10n list. Actually the most important thing is to shout in case you find show stoppers. File issue and present them to the releases@openoffice.org list
Dec 16 18:53:57 RQ	we do litmus tests with mozilla products before releases 
Dec 16 18:54:10 RQ	(well, that's how we call them)
Dec 16 18:54:42 RQ	KAMI_Work, it's a piece of the art compared to TCM :)
Dec 16 18:54:57 RQ	s/the//
Dec 16 18:55:10 KAMI_Work	RQ do you know the adress ot it?
Dec 16 18:55:17 RQ	litmus.mozilla.org
Dec 16 18:56:01 rafaella	3) set the status as APPROVED in qatrack
Dec 16 18:56:03 KAMI_Work	https://litmus.mozilla.org/
Dec 16 18:56:08 KAMI_Work	RQ :D
Dec 16 18:56:36 RQ	yes
Dec 16 18:56:38 RQ	https
Dec 16 18:56:39 Mechtilde	rafaella, I hope all peole here read this
Dec 16 18:56:43 rafaella	4) get an approved build distributed to the mirror network
Dec 16 18:56:59 thorstenziehm	RQ : I want to expand QUASTE to run also manual testing and put in the results for builds and releases. Then it will be like litmus. But a realistic time frame for this is mid/end of 2009, sorry to say. When it is needed beside TCM.
Dec 16 18:57:29 rafaella	Mechtilde, it's all documented: http://wiki.services.openoffice.org/wiki/Testing_my_translation http://wiki.services.openoffice.org/wiki/Release_Action_List_for_QA
Dec 16 18:57:43 KAMI_Work	thorstenziehm: It is not possible to adopt the mozzila program/site to my reqs?
Dec 16 18:58:03 Mechtilde	rafaella, the problem is that only few teams use it
Dec 16 18:58:04 RQ	thorstenziehm, litmus doesn't run manual tests. It's a place where testers put their testing results
Dec 16 18:58:29 rafaella	Mechtilde, not sure I understand ... use what?
Dec 16 18:58:43 Mechtilde	rafaella, use qatrack
Dec 16 18:58:59 thorstenziehm	KAMI_WORK : do we want to have more tools to collect results? I want to have one tool, where I can see on one page, how the quality status of the product (perhaps buidl is)
Dec 16 18:59:02 rafaella	Mechtilde, do you have an ide why?
Dec 16 18:59:20 artietee	http://quaste.services.openoffice.org/
Dec 16 18:59:28 Mechtilde	may be many teams don't know the release process
Dec 16 18:59:45 thorstenziehm	RQ: but litmus is also the place where the test are stored, that's what I want for Quaste also (test case specifications, and test cases)
Dec 16 18:59:56 Mechtilde	rafaella, and they don't know that they are responsible for the release
Dec 16 18:59:56 sophi	thorstenziehm: I think it's a different topic, and the second part of the meeting : to improve testing on the builds before RC
Dec 16 19:00:14 rafaella	so we need to evangelize more the release process...
Dec 16 19:00:20 sophi	Mechtilde: yes, you're right
Dec 16 19:00:25 Mechtilde	rafaella, yes
Dec 16 19:00:39 artietee	+1
Dec 16 19:00:40 RQ	thorstenziehm, oh, I thought TCM does that
Dec 16 19:00:41 sophi	rafaella: yes, a lot more communication
Dec 16 19:01:02 Mechtilde	after the 3.0 Release I have many discussion why a build wasn't release still now
Dec 16 19:01:24 RQ	yes
Dec 16 19:01:38 KAMI_Work	thorstenziehm: in QUESTe the Hungaria should be Hungarian :D
Dec 16 19:01:44 sophi	rafaella: all the documentation is there, but it is not enough
Dec 16 19:01:47 Mechtilde	rafaella, in one lang a build was released in another lang not
Dec 16 19:02:00 RQ	by the way, there was a mention on the list, that someone wrote the mirrorring guys about putting ALL localized builds into /localized/
Dec 16 19:02:08 RQ	but there was no outcome from that
Dec 16 19:02:28 thorstenziehm	RQ : yes TCM does it, but it's an additional tool
Dec 16 19:02:28 RQ	I wonder why
Dec 16 19:02:35 Mechtilde	RQ, some are in localized and some are in extended/localized
Dec 16 19:02:51 Mechtilde	RQ, because of mirror resources
Dec 16 19:02:53 RQ	Mechtilde, yeah, can't we move them out from extended? 
Dec 16 19:03:09 thorstenziehm	RQ: and TCM does not have the full functionality we need for test case specifications we have for OOo
Dec 16 19:03:25 RQ	Mechtilde, exactly. But the point is that mirrors have more resources now than  they had before
Dec 16 19:03:36 RQ	plus, it's always possible to set up a custom mirror
Dec 16 19:03:38 RQ	mine is custom
Dec 16 19:03:39 thorstenziehm	KAMI_WORK: I let is changed, thanks for the hint.
Dec 16 19:03:42 Mechtilde	RQ, but not all mirrors
Dec 16 19:04:03 RQ	Mechtilde, those who can't, could set up a custom mirror
Dec 16 19:04:07 RQ	like I said
Dec 16 19:04:19 RQ	I'm a mirror admin myself, and I know it's not hard :)
Dec 16 19:04:25 rafaella	how long have teams been using quaste now?
Dec 16 19:04:33 RQ	in fact, maintaining a mirror is easier than doing QA :)
Dec 16 19:04:34 Mechtilde	RQ, that should dicuss which the mirror team
Dec 16 19:04:47 Mechtilde	let us do it after this meeting
Dec 16 19:04:58 sophi	rafaella: may be 2 or 3 years ?
Dec 16 19:05:15 RQ	Mechtilde, the thing is, that idea about putting all localized resources into /localized/ never got suggested for the mirror admins
Dec 16 19:05:23 Mechtilde	rafaella, for the German team 1/2 year
Dec 16 19:05:34 sophi	rafaella: oups sorry no, I speak about QATrack
Dec 16 19:05:45 rafaella	sophi, I was wondering a bit....
Dec 16 19:05:56 artietee	Dutch team never used it so far
Dec 16 19:06:26 Mechtilde	artietee, why
Dec 16 19:06:26 sophi	rafaella: FR team do not use it, results are uploaded on Doc&File part of the site
Dec 16 19:06:33 rafaella	Ok. I would be interested to know if Quaste really can replace TCM. 
Dec 16 19:07:07 artietee	I'm afraid I just heard about it for the first time 
Dec 16 19:07:09 Mechtilde	rafaella, only if it covers also manual test results
Dec 16 19:07:12 Thalion72	rafaella: anything could repllace TCM .. if you put enough developer resources in
Dec 16 19:09:13 rafaella	Ok but it seems that only once Quaste covers manual tests we could consider Quaste to replace TCM
Dec 16 19:09:40 thorstenziehm	I want to remind every L10N team to upload the test results to QUASTE. We will analyze the results and try to identify the problems.
Dec 16 19:09:59 rafaella	I am asking becuase there is a new updated version of TCM with added functionality. Not sure at this point if it is worth while migrating to the newer version...
Dec 16 19:09:59 Thalion72	rafaella: yes - and (imho) manual tests are a completely different story than autmoated tests
Dec 16 19:10:04 artietee	thorstenziehm: results from automated testing?
Dec 16 19:10:13 thorstenziehm	rafaella : you are right
Dec 16 19:10:22 sophi	for those who don't know about QAste, it's here http://quaste.services.openoffice.org/index.php
Dec 16 19:10:25 thorstenziehm	artietee : yes
Dec 16 19:10:57 Thalion72	thorstenziehm: it would encourage people to upload results, if there was some feedback, that someone is analyzing those results, that are already there
Dec 16 19:11:12 rafaella	thorstenziehm, how do you include quaste in the current release process?
Dec 16 19:11:33 thorstenziehm	rafaella : I do not have the resources to integrate manual testing in QUASTE soon, so the new version of TCM should be used till then
Dec 16 19:12:37 KAMI_Work	thorstenziehm: QUASTE is opensource?
Dec 16 19:12:44 thorstenziehm	rafaella : all test results of automated testing are integrated in QUASTE automatically (for OOo and SO builds we used)
Dec 16 19:12:47 rafaella	thorstenziehm, ok. thanks
Dec 16 19:12:59 thorstenziehm	KAMI_WORK : yes, since OOoCon
Dec 16 19:13:00 sophi	ant the manual tests Thorsten is speaking about are here http://qa.openoffice.org/ooQAReloaded/TestcaseSpecifications/
Dec 16 19:13:03 KAMI_Work	if yes may independent developers might can help
Dec 16 19:13:05 Thalion72	KAMI_Work: yes .. source is in ooo cvs 
Dec 16 19:13:31 thorstenziehm	KAMI_Work : yes, every helping hand is helpful
Dec 16 19:13:52 Stefan___b	Automated Testing is included in Office build process, so that the Tests are up-to-date with all UI changes. TCM does not have a formal diredt link to changes in Office.
Dec 16 19:14:03 KAMI_Work	thorstenziehm: will you test other languages like Hungarian? And what is up whit non sun languages?
Dec 16 19:14:34 Stefan___b	Technically, QUASTE can "eat" all languages it is fed with.
Dec 16 19:14:44 KAMI_Work	:D
Dec 16 19:14:55 Thalion72	Stefan___b: but it cannot eat all platforms
Dec 16 19:15:21 Stefan___b	YET. To all limitations. a matter of ressources... :-)
Dec 16 19:15:28 Thalion72	e.g. no Mac PPC, no 63bit builds ...
Dec 16 19:15:36 Thalion72	oops 64bits
Dec 16 19:15:37 Stefan___b	65bit...
Dec 16 19:15:41 KAMI_Work	But I see lot of errors there
Dec 16 19:15:45 thackert	Stefan___b: LOl
Dec 16 19:16:02 thorstenziehm	QUASTE can be expanded easily. I do not know the technical details, but it's software all is possible.
Dec 16 19:16:49 thorstenziehm	Thalion72 : Currently we have one platform for MAC and other OS. When 64bit is needed and different MAC or Linux builds are needed, it can be integrated.
Dec 16 19:17:22 Thalion72	thorstenziehm: *we* (as the OOo community) provide Mac PPC.. and Linux64bit 
Dec 16 19:17:34 Thalion72	so - yes - *we* need these platforms
Dec 16 19:17:57 JiHui	I'm sorry. I have to go.
Dec 16 19:18:05 thorstenziehm	Thalion72 : Does QUASTE show errors with different languages or doesn't support the test script the language. I think the second one is the case.
Dec 16 19:18:08 JiHui	It's too late for me. it's 3:17 am.(I waked 6:00 am yesterday -_-;)
Dec 16 19:18:14 Stefan___b	JiHui: Good night. Or morning?
Dec 16 19:18:22 JiHui	Good-bye everyone. see you. :)
Dec 16 19:18:24 rafaella	JiHui, thanks your attending!
Dec 16 19:18:26 Thalion72	thorstenziehm: you mean error description?
Dec 16 19:18:32 sophi	back to the topic, we speak about manual testing and QUASTE because QA need more testers in an earlier stage
Dec 16 19:18:32 Mechtilde	http://quaste.services.openoffice.org/index.php?option=com_quaste&task=main&major=9 these are the result of the m13 for 3.0.1
Dec 16 19:18:39 sophi	JiHui: thanks and bye!
Dec 16 19:18:47 JiHui	I'm sorry. I want to stay but.. ^^;
Dec 16 19:18:49 JiHui	bye`
Dec 16 19:18:57 *	JiHui (n=X-iRc@211.228.29.125) has left #ooonlc
Dec 16 19:20:06 Thalion72	thorstenziehm: I see german error descriptions in QUASTE - don't know if QUASTE translates the messages if these are already in the result files
Dec 16 19:20:18 thorstenziehm	Thalion72 : If you want to work with QUASTE, we will bring the different platforms in it.
Dec 16 19:20:43 Thalion72	thorstenziehm: plese - at least MacPPC and Linux64bit
Dec 16 19:20:52 thorstenziehm	Thalion72 : The error messages are taken from the result files.
Dec 16 19:22:04 thackert	And MacIntel and MacPPC as well as Deb64 and RPM64 ... ;)
Dec 16 19:22:06 Stefan___b	Thalion72: We will squeeze an estimation from Helge :-)
Dec 16 19:23:03 sophi	all - would you have the resource to do testing on dev snapshots ? manual testing or automated testing ?
Dec 16 19:23:31 rafaella	Time is flying and in 10 minutes at latest I need to go. Shall we summarize this meeting and define what still need to be discussed?
Dec 16 19:23:34 Thalion72	sophi: on some - yes (we can'T stop people from testing ;) )
Dec 16 19:23:49 sophi	Thalion72: ah great :)
Dec 16 19:24:11 Stefan___b	It is not Christmas yet, but QUASTE is an imortant thing to have many people contribute to. (Test results production, test result processing, reviewing, discussing, planning.....)
Dec 16 19:24:14 Thalion72	sophi: but only one or two do this on a regular basis and provide feedback
Dec 16 19:24:22 Mechtilde	I use the dev snapshots for my daily work if it is possible
Dec 16 19:24:30 thackert	rafaella: Will we continue this discussion on a later point in the future?
Dec 16 19:24:46 Thalion72	Stefan___b: so please remove the "one result from one tester per language / platform / milestone) first ;)
Dec 16 19:25:02 thackert	Mechtilde: Me too ... ;)
Dec 16 19:25:12 sophi	Mechtilde: yes, I know you're doing this :)
Dec 16 19:25:28 Stefan___b	Thalion72: Please file an issue for HDE for that.
Dec 16 19:25:30 RQ	hm
Dec 16 19:25:35 Thalion72	rafaella: do you want to summarie now or rather someone to provide minutes?
Dec 16 19:25:40 RQ	is it a big problem if some teams would only do release tests?
Dec 16 19:25:44 rafaella	Well, oyou feel that there is a need for another meeting, I am happy to ammounce a follow up meeting in January.
Dec 16 19:25:54 Mechtilde	RQ, yes
Dec 16 19:26:00 Thalion72	RQ: no - as longs as some other teams do more extensive testing
Dec 16 19:26:09 RQ	okay
Dec 16 19:26:10 Stefan___b	RQ: QA constantly run TestTool to "secure" the basic code. Means: Mostly English UI office is built and tested here. 
Dec 16 19:26:25 thackert	rafaella: Yes please ... I think we miss your second point today, or am I wrong?
Dec 16 19:26:36 rafaella	Thalion72, I think that a summary of Action Items would be ok
Dec 16 19:26:40 Mechtilde	rafaella, I think it is helpfull to continue
Dec 16 19:26:42 Thalion72	thackert: you are right ;)
Dec 16 19:26:43 RQ	Stefan___b, but do English builds differ so much at binary level?
Dec 16 19:26:55 Stefan___b	It should not!
Dec 16 19:27:02 Mechtilde	but....
Dec 16 19:27:32 Thalion72	RQ: not much .. but the small differences might cause tricky errors
Dec 16 19:27:36 rafaella	Mechtilde, unfortunately I I cannot continue today
Dec 16 19:27:46 Mechtilde	not today rafaella 
Dec 16 19:27:58 Stefan___b	RQ: But even translation of paragraph style names did cause crashes in the early days...
Dec 16 19:28:06 RQ	hm... 
Dec 16 19:28:21 RQ	but this can be cought during release tests
Dec 16 19:28:28 Mechtilde	rafaella, we should continue next year
Dec 16 19:28:31 Thalion72	RQ: hmm .. not very likely
Dec 16 19:28:35 RQ	uh
Dec 16 19:28:54 Stefan___b	two names translated into one string the code could not deal with... A coding problem.
Dec 16 19:28:57 sophi	RQ: QA team needs really more help, so the more people are testing them, the more we can catch bugs early
Dec 16 19:28:58 Thalion72	RQ: the problem is, that we will never ever find *all* errors before a release
Dec 16 19:29:04 RQ	cause I'm not sure how much tests my testers are willing to perform :)
Dec 16 19:29:04 Stefan___b	Too late to fix in RC time.
Dec 16 19:29:09 rafaella	Mechtilde, yes, sure.
Dec 16 19:29:22 RQ	a few did very wel, but most haven't even started yet
Dec 16 19:29:39 Stefan___b	So localized builds must be tested during development, too.
Dec 16 19:29:41 Thalion72	RQ: but the more people help, the more likely it is, that we find the real big bugs
Dec 16 19:29:53 RQ	yeah, i know
Dec 16 19:30:13 thorstenziehm	Thalion72, RQ: Let's find them :-)
Dec 16 19:30:39 Thalion72	btw .. no bug-hunting-session this month?
Dec 16 19:30:52 Stefan___b	Thalion72: BIG problems can never be from small code lines... :-)
Dec 16 19:30:59 thackert	Thalion72: That was my question, too ... ;)
Dec 16 19:31:11 Stefan___b	Thalion72: I was about to announce: January 13 will be it.
Dec 16 19:31:13 Thalion72	Stefan___b: uhm .. yess .. your best today ;)
Dec 16 19:31:16 RQ	Stefan___b, yeah, they can, if the line is crash(); :)
Dec 16 19:31:27 sophi	Thalion72: under the christmas tree ;)
Dec 16 19:31:54 thorstenziehm	rafaella : summarize?
Dec 16 19:32:40 rafaella	who is volunteering to create an FAQ wiki for Testool questions?
Dec 16 19:33:16 thorstenziehm	I can create the page tomorrow and I will annouce it
Dec 16 19:33:23 rafaella	I volunteer to announce the next meeting in January
Dec 16 19:34:11 sophi	thorstenziehm: rafaella: thank you :)
Dec 16 19:34:15 rafaella	Any other action items that came up during the meeting?
Dec 16 19:34:39 sophi	rafaella: evangelize the release process for RC
Dec 16 19:34:40 Thalion72	remove testtool test cases from release sanity in TCM?
Dec 16 19:34:51 sophi	Thalion72: +1
Dec 16 19:34:59 thackert	Thalion72: +1
Dec 16 19:35:09 thorstenziehm	Thalion72: +1
Dec 16 19:35:18 Thalion72	ok, will do
Dec 16 19:35:28 rafaella	Thalion72, perfect!
Dec 16 19:35:30 Stefan___b	Thalion72: +1, do I get it right that all l10n teams must do that seperately?
Dec 16 19:35:32 sophi	Thalion72: thanks :)
Dec 16 19:36:07 sophi	Stefan___b: on a willing basis 
Dec 16 19:36:21 rafaella	Thank you very much for attending the meeting, have a great holiday break and see you on IRC next year!
Dec 16 19:36:46 Stefan___b	sophi: But all must know that this shall be the case. 
Dec 16 19:36:49 Thalion72	.. done :)
Dec 16 19:36:51 thackert	rafaella: Same to you :) I hope to have the time to join the next meeting ...
Dec 16 19:36:53 sophi	rafaella: thank you and nice holiday for you too
Dec 16 19:37:03 sophi	Stefan___b: yes
Dec 16 19:37:05 thorstenziehm	bye
Dec 16 19:37:12 artietee	bye
Dec 16 19:37:15 Stefan___b	rafaella: thanks for getting us together :-)
Dec 16 19:37:42 RQ	hm
Dec 16 19:37:51 *	thorstenziehm (n=chatzill@nat/sun/x-a0e0ed945e7c1228) has left #ooonlc
**** ENDING LOGGING AT Tue Dec 16 19:38:12 2008
